Blog about better using AI Infrastructure via MIG - #252
Merged
Conversation
Deploying website with
|
| Latest commit: |
737508d
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://9ddee9e4.website-dab.pages.dev |
| Branch Preview URL: | https://utho-mig-blog.website-dab.pages.dev |
…grams - Open with the GPU sharing problem and the time-slicing / MPS / MIG spectrum so MIG has a story, not just a definition - Add complete step-by-step nvidia-smi walkthrough for one GPU: check, enable, list profiles/placements, carve (incl. mixed geometry), verify, run on a slice, destroy CIs/GIs, disable MIG - Add the all-8-GPUs variant of every command plus fleet-wide teardown - Add GPU Operator install command and mig.strategy single/mixed explanation, plus declarative disable via all-disabled label - Replace dark AI-generated infographics with light Excalidraw-style diagrams (cover, sharing spectrum, GI/CI, k8s GPU stack, runtime flow) - Regenerate feeds Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
|
Pushed a rework of the post directly to this branch (great groundwork @shkatara, the real captured outputs and pitfalls carried straight over). What changed:
|
Ran the full MIG lifecycle on the 8x RTX PRO 6000 node (103.209.144.201):
- Real -lgip table (full profile list incl. +gfx/+me.all, 95.12 GiB for 4g.96gb)
- Real -lgipp placements: 12-unit grid, {0,3,6,9}:3 for 1g.24gb; rewrote
the placement explanation accordingly
- Real -lgi/-lci (placements 0:3/3:3/6:3/9:3), mixed-geometry capture
(2g.48gb at 0:6 + two 1g.24gb), dgi-before-dci failure message
- Real enable/disable outputs incl. persistence-mode warnings, all-8 bus IDs
- Real CUDA smoke test on a slice UUID (torch cu130, 24192 MiB visible)
- Fix prereqs: 256 cores
- Mark post as draft until final review; feeds regenerated (drafts excluded)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
|
Update: all command outputs are now real captures from the box (ran the full lifecycle on the 8x RTX PRO 6000 node today, with Saiyam's approval for briefly pausing the HAMi demo pods). What the live run corrected vs the reconstructed draft:
The post is marked |
- New baseline subsection: what a CUDA context is, how kernels land on SMs, and the one-context-at-a-time rule that makes GPUs single-tenant - Time-slicing reframed as contexts taking turns, with concrete OOM and noisy-neighbor examples - MPS reframed around its actual trick: merging all clients into the MPS server's single context for real space-sharing, incl. the opt-in limit env vars and the shared-fault-domain failure story - MIG subsection extended with L2/bandwidth partitioning and per-slice contexts; comparison table gains "the trick" and parallelism rows - Spectrum diagram corrected: time-slicing shows separate contexts taking turns, MPS shows the merged context Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- GPU context switches spill per-SM register files and shared memory to VRAM, orders of magnitude heavier than a CPU thread switch - Pre-Pascal kernels were uninterruptible (kernel-boundary switching only); Pascal added instruction-level compute preemption, which k8s time-slicing relies on today - Short kernels still interleave at kernel boundaries; preemption is the expensive escape hatch for long kernels - Add training vs inference sharing intuition Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uance Cross-checked against the GPU-Enabled Platforms on Kubernetes book: - Time-slicing: add the no-fairness-guarantee point (no fixed quantum, no priorities; honest name is GPU oversubscription) - Memory: every context sees the full VRAM as allocatable and nothing tracks intended shares - MPS: Volta+ clients keep separate GPU address spaces; shared fate is the scheduler and the server, not raw memory access Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- frontmatter `authors:` array (first entry primary), legacy `author:` still works everywhere - post page renders all authors: stacked byline avatars, joined names, one bio card each, metadata + OG + JSON-LD author arrays - author pages and counts include co-authored posts - blog cards show all author names and avatars Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Technical review fixes: - 1g.24gb slice is 46 SMs, not "roughly 47" (matches the real -lgip table; driver reserves a few, 4x46=184 of 188) - media-engine profiles: plain slices already carry a proportional NVDEC/NVENC; -me strips them, +me.all hoards all, +gfx adds graphics Utho sponsor feature (frontmatter-driven, only shows when `sponsor:` set): - SponsorCallout component: slim "Hardware partner" strip near the top, thank-you CTA card at the end; rel="sponsored" links - credit Utho on this post (URL is a placeholder pending the UTM link) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- set sponsor.url to the Kubesimplify UTM link
- SponsorCallout CTA shows the Utho logo (light/dark swap via .dark),
falling back to the text name when no logo is set
- placeholder wordmark SVGs at public/img/sponsors/utho-{dark,light}.svg
to be replaced with the official assets (same filenames)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- add utho-logo-light.png (navy, light theme) + utho-logo-dark.png (white, dark theme), resized to 600px wide, transparency preserved - point frontmatter at the real assets; drop placeholder wordmark SVGs Note: Utho names by theme (dark = white ink for dark bg), so mapping was verified by pixel color, not filename. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the slim top strip with the same logo CTA card used at the end, so the sponsor is credited identically above and below the article. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.